feat: Add filter by UID feature by using existing --filter syntax - #3237

It's intended to be used for debugging purpose. (launchSetting.json or CI input parameter)

Existing filter syntax match filterName or fullName.
So it can't distinguish BenchmarkCases that have different job.

Example

[InProcess]
[Config(typeof(DebugInProcessConfig))]
public class Benchmarks
{
    [Benchmark]
    public void Benchmark01() {}
}

And when filtering benchmarkCase with fullName.
It needs double-quote for filter value (When Params exists empty space is included on fullName)

To run specific benchmark for debugging purpose,
It's easier to specify BenchmarkCase by uid that are retrieved by --list json command.

It's intended to be used for debugging purpose. (launchSetting.json or CI input parameter)

Existing filter syntax match filterName or fullName. So it can't distinguish BenchmarkCases that have different job.

Thanks. In such case I would expect the user to modify the config and just comment out the Job they don't want to use.

Thanks. In such case I would expect the user to modify the config and just comment out the Job they don't want to use.

I want to avoid modify/rebuild benchmark if possible.
A more specific example would be testing across multiple .NET versions, as shown below:

I want to run specific benchmark case(Job) by editing launchSettings.json.

[SimpleJob(RuntimeMoniker.Net80)]
[SimpleJob(RuntimeMoniker.Net90)]
[SimpleJob(RuntimeMoniker.Net10_0)]
public class MyBenchmark
{
    [Benchmark]
    public void Test()
    {
        // benchmark
    }
}

If reusing the --filter option is not desirable,
I want other options to execute specific benchmark case.

I want to avoid modify/rebuild benchmark if possible.

This is a great goal and that is why we should encourage users to specify all the jobs arguments via command line arguments.

My personal approach is to always have a config with the default settings I usually want (like 1 warmup iteration), with a single job marked AsDefault and all the exporters and then everything else (diagnosers, filters, tfms) specified via command like arg.

But in this particular example we talk about Debugging, which requires and IDE (at least for most of us). So I would expect the users to simply go and modify the code by commenting out the job they don't like just a moment before they hit F5.